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Abstract 

We introduce the concept of attainable sets of payoffs in two-player repeated games 
with vector payoffs. A set of payoff vectors is called attainable if Player 1 can ensure 
that there is a finite horizon T such that after time T the distance between the set and 
the cumulative payoff is arbitrarily small, regardless of what strategy Player 2 is using. 
This paper focuses on the case where the attainable set consists of one payoff vector. 
In this case the vector is called an attainable vector. We study properties of the set of 
attainable vectors, and characterize when a specific vector is attainable and when every 
vector is attainable. 
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1 Introduction 



There are various dynamic situations in which the stage payoff is multi-dimensional, and 
the goal of the decision maker is to drive the total vector payoff as close as possible to a 
given target set. One such example is dynamic network models, which include a variety of 
different logistic applications such as production, distribution and transportation networks. 
In the literature on dynamic network flow control [21 El [U EJ H7] , the supplier tries to meet 
a multi-dimensional demand. His goal is to ensure that the difference between the total 
demand and the total supply converges with time to a desirable target. One can model 
such a situation as a two-player repeated game, where Player 1 is the decision maker, and 
Player 2 represents the adversarial market that controls the demand. For instance, in the 
distribution network scenario, the supplier has a desirable multi-dimensional inventory level 
that he would like to maintain, despite erratic behavior of the demand side. Having to deal 
with an adversarial opponent requires the supplier to cope with the worst case possible. This 
motivates our main objective: to find conditions that characterize when a specific target 
vector x can be attained against any possible demand pattern exhibited by the market. 

A second example is the Capital Adequacy Ratio. The third Basel Accord states that 
(a) the bank's Common Equity Tier 1 must be at least 4.5% of its risk-weighted assets at 
all times, (b) the bank's Tier 1 Capital must be at least 6.0% of its risk-weighted assets at 
all times, and (c) the total capital, that is, Tier 1 Capital plus Tier 2 Capital, must be at 
least 8.0% of the bank's risk weighted assets at all times. To accommodate this example 
in our setup, consider the following 3-dimensional vector. The first coordinate stands for 
the per-period difference between the bank's Tier 1 Capital and 6.0% of its risk-weighted 
assets; the second coordinate stands for the per-period difference between the bank's Tier 
1 Capital and 6.0% of its risk-weighted assets; and the third stands for the per-period 
difference between the total capital and 8.0% of the bank's risk weighted assets. According 
to the Capital Adequacy Ratio the coordinates of this vector should be non-negative. Here, 
Player 1 represents the bank's managers who control its assets, and Player 2 represents 
market behavior, which is unpredictable and thought of as adversarial. Thus, the goal of 
Player 1 is to design a strategy that would drive the 3-dimensional total payoff to the target 
set - the non- negative orthant. To ensure that they fulfill the requirements of the Basel 
Accord, banks try to hold a capital buffer on top of the regulatory minimum, and they 
periodically adjust their assets to be at the top of the buffer [T5"l l2~T]. 

To model such situations we study two-player repeated games with vector payoffs in 
continuous time. We say that a set A in the payoff space is attainable by Player 1 if there is 
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a time T such that for every level of proximity, e > 0, Player 1 has a strategy guaranteeing 
that against every possible strategy for Player 2, the distance between A and the cumulative 
payoff up to any time t greater than T is smaller than e. When a set A is attainable, Player 
1 can plan his actions, based on historical inventory and market data, to ensure that the 
inventory level would converge to A. In the case where A consists of one point, the supplier 
may guarantee that his inventory level will be as close as he wishes to an ideal level. 

The definition of attainability is close in spirit to the concept of approachable sets [6], 
which refers to the average stage payoff rather than the cumulative payoff. While a set A 
is attainable by Player 1 if he can ensure that the cumulative payoff converges to it, A is 
approachable by him if he can ensure that the average payoff converge to the set. 

In case a point, say x, is attainable by Player 1, it implies that the long run inventory 
level is stable around x. On the other hand, if x is approachable, it merely guarantees that 
the average payoff converges to x. This may happen even when the actual payoff does not 
converge to x, and, in fact, even when any fixed running average does not converge to x. 
This observation suggests that although the notions of attainability and approachability 
look similar, and indeed are related, the flavor of the results and their proofs are completely 
different. 

When a vector x is attainable by a player, the player can ensure that the total payoff 
converges to that vector. In particular, he can ensure that the total payoff gets close to x, 
and remains around x forever. This implies that once some vector x is attainable, so it the 
vector 0. The first main result, Theorem [lj characterizes when is attainable. The result 
implies that, with regard to the vector 0, attainability and approachability coincide: the 
vector is attainable by a player if and only if it is approachable by him in the game in 
discrete time. 

The second main result, Theorem[2j characterizes when a given vector x is attainable. It 
turns out that a vector x is attainable by a player if and only if (i) the vector is attainable 
by him, and (ii) the vector is attainable by him in a game whose payoffs are translated 
by 5x for some 5 > 0. Condition (i) is shared by all x: in order for any x to be attainable, 
the vector must be attainable. Condition (ii), on the other hand, is point specific, and it 
uses the attainability of in a modified version of the original game. 

The last result, Theorem |3j characterizes the cases in which the controller has full 
freedom in setting his target level, in the sense that any vector x is attainable. 

There is a literature on decision problems related to dynamic multi-inventory in con- 
tinuous time (see for instance, the continuous-time control strategy in [8]). The control 
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literature up to this point refers to one-person (the controller) decision problems facing 
uncertainty. As far as we know, this paper is the first taking a strategic approach to the 
problem. 

In the game in continuous time we discuss, players are allowed to use a special type of 
behavior strategies. These strategies are characterized by an increasing sequence of positive 
real numbers, that divide the time span [0, oo) into sub-interval. The play of the player 
in each interval depends on the play of the other player before the interval starts, but is 
independent of the play of the other player during that time interval. This is equivalent to 
saying that before the game starts, a player sets an alarm clock to ring in certain times, 
and whenever the clock rings, the player looks at the historical play path up to that point 
and determines how to play until the next time the clock rings. We later discuss the 
interpretation of this type of strategies. 

This paper is organized as follows. In Section [2j we provide motivating examples. In 
Section [3] we introduce the model and main definitions. In Section [4j we introduce and 
discuss the results. Section [5] is devoted to the discussion of a few aspects related to the 
definition of attainability and to the type of continuous strategies that we are using. Proofs 
are relegated to Section [6j 



2 Motivations 

This section details one motivation of our study: distribution networks. Consider a dis- 
tributer of a certain product who has two warehouses A and B in different regions. Every 
month the distributer can order products from factories to each of the warehouses, and he 
can transport products between the two warehouses, while vendors order products from the 



warehouses. This situation is described graphically in Figure 1(a) 
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(a) Three distribution flows /a, /t, 
Jb and two vendors requests wa, wb- 




(b) Factory manager can sell directly to 
vendors: node C represents factory. 



Figure 1: Distribution network with warehouses A and B. 



In Figure 1(a), Ja and fs are the number of products that are sent from factories 
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to the two warehouses A and B, fx is the number of products that are transported from 
warehouse A to warehouse B, and wa and wb are the number of products sent from the two 
warehouses to vendors. Negative flows are interpreted as flows in the opposite direction; 
e.g., if vendors return products to warehouse A (resp. to warehouse B), then wa (resp. 
wb) is negative. If products are transported from warehouse B to warehouse A, then fx 
is negative. We analyze this situation in continuous time. The change of stock in the two 
warehouses is given by the 2-dimensional vector 



u(a 1; a 2 ) 



1-10 
Oil 



It 

v/w 



Wz 



where a\ = (f A , f B , f T ) is the decision variable of the distributer, and a 2 = {w A , w B ) is the 
uncontrolled market demand at time t. 

Suppose that the number of products that can be ordered by vendors at each time 
instance is bounded by 2, and the number of products that can be returned by vendors 
to each warehouse at every time instance is 3. In other words, w A and w B are in [—3,2]. 
Suppose also that the amount of product that the distributer can order from or return to 
the factories and transport between the two warehouses is bounded by 5. 

This situation can be described by a two-person game as follows. The distributer (Player 
1) has 8 actions 

(5, 5, 5), (5, 5, -5), (5, -5, 5), (5, -5, -5), (-5, 5, 5), (-5, 5, -5), (-5, -5, 5), (-5, -5, -5), 

while the market demand or nature (Player 2) has 4 actions 

(-3, -3), (-3, 2), (2, -3), (2, 2). 

The payoffs correspond to the change of stock in the two warehouses, and are given by the 
following table: 
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(-3,-3) 


(-3,2) 


(2,-3) 


(2,2) 


(5,5,5) 


(3,13) 
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(-2,13) 


(-2,8) 


(5,5,-5) 


(3,3) 


(3,-2) 
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(-2,-2) 
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(13,3) 
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(8,-7) 
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(-7,13) 


(-7,8) 
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(-7,3) 
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(-12,3) 
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(-5,-5,5) 


(3,3) 


(3,-2) 


(-2,3) 


(-2,-2) 


(-5,-5,-5) 


(3,-7) 


(3,-12) 


(-2,-7) 


(-2,-12) 



Figure 2: The strategic-form game corresponding to the situation. 

per-period At every time instance the two players choose their actions. Each market 
behavior translates into a mixed action of Player 2, and each behavior of the distributer 
corresponds to a mixed action of Player 1. The (2-dimensional) total payoff up to time 
t is the number of products that are stored in each of the two warehouses. The goal of 
the distributer is to ensure that the total number of products in each warehouse does not 
exceed its capacity, that is, that the total payoff should not exceed a certain (2-dimensional) 
bound. 



Figure 1(b) describes the case where the factory manager can sell directly to vendors, 
bypassing the distribution to warehouses. This situation can be represented by adding an 
additional node C modeling the factory, and an edge that represents the market demand. 
The stock is now a 3-dimensional vector, as we have to take into account the inventory 
available at the factory, and consequently the change in the stock modifies as shown below: 
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A recurrent question in the network flow control literature (3j EJ |8j El H7j is about 
conditions that ensure the existence of a control strategy that drives the excess supply 
vector to a desired target level in W 71 regardless of the unpredictable realization of the 
demand. The equivalence between the excess supply and the cumulative payoff in the 
dynamic game motivates our study. The rest of the paper is devoted to the analysis of 
conditions under which Player 1 has a strategy ensuring the attainability of a given point, 
regardless of the behavior of Player 2. 



6 



Situations where the target is to control the total payoff occur also in production and 
transportation networks. Production networks describe production processes and activities 
necessary to turn raw materials into intermediate products and eventually into final prod- 
ucts. The nodes of the networks represent raw materials and intermediate/final products. 
The buffer at each single node % models the amount of material or product of type i stored 
or produced up to the current time, and hyper-arcs describe the materials or products con- 
sumed (tail nodes) and produced (head nodes) in each activity or process. Transportation 
networks model the flow of commodities, information, or traffic; nodes of the networks rep- 
resent hubs and the buffers at the nodes describe the quantity of commodities present in 
the hubs. The edges describe transportation routes. 

2.1 Related control and approachability literature 

We highlight two main streams of related literature, one from the control area and the 
second from the approachability area. These two bodies of literature have in common the 
interest towards robustness. 

Connections between robust control and noncooperative game theory has a long history 
(see, e.g., [1]). Robust control is the area of control theory that looks for strategies that 
"control" the state of a dynamical system, for instance, drive it to a given set, despite 
the effects of disturbances (see the seminal paper pj). Among the foundations of robust 
control we find two main notions that can be related to attainability and are surveyed in 
[7]. The first notion, robust global attractiveness, refers to the property of a set to "attract" 
the state of the system under a proper control strategy, independently of the effects of the 
disturbance. The second notion, robustly controlled invariance, describes the property of 
a set to bound the state trajectory under a proper control strategy, independently of the 
effects of the disturbance. Both notions are widely exploited in a variety of works that 
contribute to the use of robust control in dynamic network flow models [21 [31 01 [H [91 [T7] . 

With regard to the second stream of literature, the definition of attainability is close 
in spirit to the concept of approachability (see [HI H21 HH [22l [231 (211 EH [26]). The 
two concepts differ in that approachability aims at controlling the average payoff whereas 
attainability aims at controlling the cumulative payoff. Approachability theory has been 
applied in several areas of game theory, such as allocation processes in coalitional games 
[T5] . regret minimization [23111], and adaptive learning [101 El ES E3] , just to name a few. 
Approachability theory has been originally formulated for discrete-time repeated games with 
finite-dimensional payoffs. Since then it has been extended to infinite-dimensional payoffs 
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|19j and to continuous-time repeated games, showing common elements with Lyapunov 
theory [13]. 

This paper focuses on the game theoretic aspects related to attainable sets. A discussion 
on applications to network flow control problems is introduced in a companion paper |22j. 

3 Attainability 

In the first part of this section we introduce the mathematical model of repeated game 
in continuous time and elaborate on the type of strategies used by the players. In the 
remaining part, we provide a formal definition of attainability. 

3.1 The model 

We study a two-player repeated game with vector payoffs in continuous time V. The set of 
players is N = {1,2}, and the finite set of actions of each player i is A{. The instantaneous 
payoff is given by a function u : A\ x Ai — > M m , where m is a natural number. We assume 
w.l.o.g. that payoffs are bounded by 1, so that u : A\ x A2 — > [—1, l] m . We extend u to the 
set of mixed-action pairs, A(^4i) x A(A2), in a bilinear fashion. The one-shot vector-payoff 
game {A\, A2,u) is denoted by G and we will say that the game in continuous time T is 
based on G. If i £ {1, 2}, then — i denotes the player who is not i. 

The game V is played over the time interval [0, 00). We assume that the players use 
non-anticipating behavior strategies with delay, which we define now. Roughly, a non- 
anticipating behavior strategy with delay divides time into blocks. The behavior of a player 
in a given block depends on the behavior of the other player up to the beginning of that 
block. In other words, the way a player plays during a given block of time does not affect 
the way the other player plays during that block. Still, it may affect the other player's play 
in subsequent blocks. 

Denote by Cj the set of all controls of player i, that is, the set of all measurable functions 
from the time space, [0, 00), to player i's mixed actions. That is, 

d := {ai : [0, 00) — > A(Aj), 04 is measurable} . 

Definition 1 A function ai : C_j — > Ci is a behavior strategy with delay (or simply a 
strategy,) for player i, if there exists an increasing sequence of real numbers {rf such 
that for every a-i^a!_ i G C—%, 

a-i(t) = a'_i(t) Vt G [0,4) (ffi(a_i))(i) = fa (OX*) ^ k > £ [0,rf +1 ), 
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where rf = 0. 

Remark 1 A strategy as we defined here is called a non-anticipating strategy with delay 
in the literature of differential games. An equivalent formulation, that may look more trans- 
parent to game theorists, is as follows. A strategy for player i is a vector (r- , af)k£N where 
(rf )fcgN is an increasing sequence of real numbers, and for each k G N, o\ is a function 
that maps play paths (of both players) on the interval [0,Tj ~ 1 ) to plays of player i in the 
interval [t* -1 , t*). 

When defining a strategy and when referring to a strategy we will usually take the equiv- 
alent formulation given in this remark. 

In the sequel we will refer to the real numbers (rf)k&i in Definition [l] as the updating 
times related to 0{. 

Remark 2 Continuous time is usually used as a convenient model for discrete time, when 
the gap between two stages is small. This is the case here as well. Suppose that time is 
discrete, and the time difference between any two decision moments is extremely small. 
Suppose that observation of the actions of the other player is time consuming and possibly 
costly, so that players cannot observe each other at every decision point. Thus, the players 
are in fact playing a game in discrete time, in which they can randomize at every decision 
point, but they observe the actions of the other player only rarely, relative to the frequency 
in which they take actions. By improving the observation technology a player can observe 
the actions of the other player more frequently, but this frequency will always be significantly 
slower than the frequency in which actions are taken. 

Every pair of strategies a = (0-1,0-2) uniquely determines a play path (a t (a))teu + - The 
payoff (vector) up to time T associated with the pair of strategies a is given by 

7 T {a)= f T u{a\(j))dt 6 R m . (1) 
Jo 

When we wish to emphasize that the payoff is in the game based on G we will write 7^ 
rather than j T . Since payoffs are bounded by 1, the integral in which is the cumulative 
payoff up to time T, is well-defined. 

3.2 Attainability: the definition 

The subject matter of the paper is the concept of attainable sets: a set of vectors is attainable 
by a player if he can guarantee that the distance between the set and the total payoff 
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converges to 0, regardless of the strategy of the other player. We provide here one definition 
for this concept. Two alternative definitions are given and discussed in Section [5] 

Definition 2 The set Y C M m is attainable by Player 1 if there is T > such that for 
every e > there is a strategy o\ of Player 1 such tha^\ 

d( 1 t {a l ,a 2 ),Y)<e, Vt > T,Va 2 . 

A set Y is attainable if there is a finite horizon T such that Player 1 can ensure, against 
any possible strategy of Player 2, that the cumulative payoff up to any time t > T is within 
e from Y. Note that the time T is uniform across all levels of precision. That is, in order 
for Y to be attainable by Player 1, Player 1 must be able to guarantee that the cumulative 
payoff at any time longer than T would be within any e from Y. However, different e's 
might require different strategies employed by Player 1. It might therefore happen that 
although Y is attainable by Player 1, the cumulative payoff would never touch Y itself. We 
say that the strategy o\ in Definition [2] attains the set Y up to e. 

When Y = {x}, that is, Y contains a single vector, we say that the vector x is attainable 
by Player 1. Denote by W the set of attainable vectors. 

The definition of an attainable set looks close in spirit to that of approachable set in 
games played over discrete set of times [6]. There is, however, a significant difference between 
the two concepts. A set is approachable if the average payoff converges to it, while a set 
is attainable if the cumulative payoff converges to the set. In other words, approachability 
refers to the convergence of the average payoff, while attainability to the convergence of the 
cumulative payoff. Indeed, the results characterizing approachable sets and attainable sets 
are significantly different, both in contents and in spirit. 

4 Results 

This section presents the three main results of our study. The first result, Theorem 1, focuses 
on the conditions under which the vector is attainable. Attainability of turns out to be 
crucial if we wish to inspect attainability of a single vector i^O, and this constitutes our 
second result (see Theorem 2). The third result builds upon the previous two results, and 
provides a stronger condition that ensures that any vector x £ M m is attainable. 

We start with a simple observation regarding the notion of attainability, which holds 
due to the continuous time setup. Its proof is differed to the last section. 

1 The distance and the norm referred to throughout the paper is ||.||2- For instance, d(x, A) — min^gy — 

vh- 
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Proposition 1 The set W is a convex cone. 

The next theorem characterizes when the vector is attainable. To state this result we 
need the following notations. Let A G M m . Denote^] by (A, G) the zero-sum one-shot game 
whose set of players and their action sets are as in the game G, and the payoff that Player 
2 pays to Player 1 is (A, u{a\, 02)) for every (01,02) G A\ x A2. As a zero-sum one-shot 
game, the game (A, G) has a value, denoted v\. 

For every mixed action p G A(A\) denote 

D 1 (p) = {u(p,q):q£A(A 2 )}. 

D\ (p) is the set of all payoffs that might be realized when Player 1 plays the mixed action p. 
If v\ > (resp. V\ > 0), then there is a mixed action p G A(A\) such that D\(p) is a subset 
of the closed half space {x G M m : (A, x) > 0} (resp. half space {x G M m : (A,x) > 0}). 

Theorem 1 T/ie following conditions are equivalent. 
Bl T/ie vector G M m is attainable by Player 1; 
B2 v x >0 for every A G M m . 

Corollary 2 in [6j implies that the vector is approachable in the game in discrete time 
with payoff function u if and only if condition B2 holds. We thus deduce the following 
corollary to Theorem [TJ 

Corollary 1 The vector G W 11 is attainable by Player 1 inT if and only if it is approach- 
able by Player 1 in the repeated game in discrete time with payoff function u. 

The following result characterizes when a given vector x is attainable. We will need the 
following notation. For every y G M m denote by (G — y) the two-player one-shot game that 
is identical to G except for its payoff function. The payoff function of {G — y) is (u — y), 
where (u — y){a\,a2) = ^(01,02) — y for every a\ G Ai and a<i G A2. 

Theorem 2 Let 7^ x G M. m . The vector x is attainable by Player 1 if and only if 

Bl The vector G M m is attainable by Player 1 

and either one of the following conditions holds: 

2 The inner product is denned by (x, y) := x iVi f° r every x, y € K m . 
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B3 There is Sq > such that for every q G A(^2) there is p G A.(Ai) and S > 6q satisfying 
u(p, q) = Sx. 

B4 There is 5 > such that the vector G M. m is attainable by Player 1 in the game based 
on (G — Sx). 

Theorem [2] implies that whenever any vector x is attainable, so is the vector 0. Since 
attainability is concerned with the cumulative payoff, once a target level is (almost) reached, 
this level should be maintained in the long run. This means that once a neighborhood of a 
target level x is reached, from that point on the level ought to be attained. This is the 
reason why is attainable when any vector x is, and why plays a major role in the theory 
of attainability. However, condition Bl alone is not sufficient for the attainability of other 
vectors. 

Condition B3 states that for every q G A(^2) there is a strategy for Player 1 such that 
the payoff u(p, q) is x multiplied by a scalar S, which is bounded away from 0. Condition 
B4 states that is attainable in some game whose payoff is a translation of the original 
payoff function by Sx. 

The following theorem deals with the case where all the vectors are attainable. 
Theorem 3 The following statements are equivalent: 
CI Every vector x G M m is attainable by Player 1; 
C2 v x > for every A G M. m \ 0. 

Remark 3 If condition C2 is satisfied, then for every open half space H of W 71 , there is 
a mixed action p G A(Ai) such that D\{p) C H. Standard continuity and compactness 
arguments imply that in this case there is S\ > such that for every half space H there is 
p G A(A\) satisfying d{D\{p),H) > S\. Stated differently, there is S2 > such that for 
every vector A whose l\-norm is 1, (X,u(p,q)) > S2 for every q G A(^2)- 

Note the difference between condition B2 of Theorem[T]and condition C2 of Theorem|3l 
In the former, the value of the scalar-payoff game with payoffs (A, u(p, q)) is non-negative for 
every direction A G M m \ 0, while in the latter it is strictly positive. The former guarantees 
attainability of the vector 0, while the latter guarantees that every vector is asymptotically 
attainable. 
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5 Discussion 



The problem setting and the results illustrated above give rise to a number of additional 
questions such as: Is Theorem 1 still valid if the repeated game is in discrete time? Can we 
derive a different notion of attainability that captures the model's behavior when the time 
of convergence to a given set is unbounded from above? What if the updating times are 
not pre-determined and can be selected as a function of the information available up to the 
previous updating time? We next elaborate on the above issues one by one and highlight a 
few other open problems which we leave for future research. 

5.1 Continuous time versus discrete time. 

The characterization presented in Theorem [T] depends crucially on the continuous time 
setting. The following example shows that this result is invalid when time is discrete. 

Example 1 Consider a game in discrete time where payoffs are one-dimensional and each 
player has two actions. Payoffs are given by the following matrix: 





L 


R 




L 


R 


u 


-2 - 1 


-2 + 1 


U 


-3 


-1 


B 


2 - 1 


2 + 1 


B 


1 


3 



Figure 3: The payoff function in Example [TJ 



The payoffs in this game are the sum of two numbers, one determined by Player 1 (-2 if 
he plays U, 2 if he plays B), and the other by Player 2 (-1 if she plays L, 1 if she plays R). 

Condition B2 is satisfied, and therefore is attainable by Player 1. The following 
strategy guarantees that the cumulative payoff is^] within 9 • 2r/ from at any t > 2; the 
details of the proof can be found in the proof of Theorem [TJ Divide the time line into 
countably many blocks, where the length of the k-th block is ?. In the k-th block Player 1 
plays U if the cumulative payoff at the beginning of the block is positive, and he plays B 
otherwise. 

We show that is not attainable by Player 1 in the game in discrete time. When time 
is discrete, a behavior strategy for a player is a function that assigns a mixed action to each 
past history. For every i 6 N let %f be the mixed action played by Player 1 at stage t. The 
mixed action p depends on past play. Let 02 be the strategy that at each stage I plays L 
if p^(U) > and R otherwise. The stage payoff is then at least 2 whenever Player 2 plays 

3 The extra 9 appears because in this example the payoffs are not bounded by 1, and 3 2 = 9. 
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R, and at most —2 whenever Player 2 plays L. In particular, if the total payoff up to stage 
t is in the interval [—5,5], then the payoff up to stage £ + 1 lies outside this interval. Thus, 
the cumulative payoff does not converge to 0. ■ 

Example [T] suggests that the characterization of the set of attainable vectors in games 
in discrete time is more challenging that the characterization in continuous time. 

5.2 Alternative definitions of attainability 

We here provide two alternative definitions of the concept of attainability, which we term 
asymptotic attainability and weak asymptotic attainability. We then explore some relations 
between the three definitions. 

For every set Y C M. m we denote by B(Y, e) the set of all points whose distance from at 
least one point in Y is less than e, that is, 

B(Y, e) := {x£l ra : d(x, Y) < e}. 

When Y is a single point x, we write B(x,e) instead of B({x},e). 

Definition 3 (i) The set Y C ]R m is asymptotically attainable by Player 1 if there is a 
strategy o\ for Player 1 such that for every strategy 0*2 of Player 2, 

lim d(j T (a l ,a 2 ),Y) = 0. (2) 

T— >oo 

(ii) The set Y is weakly asymptotically attainable by Player 1, if the set B(Y,e) is asymp- 
totically attainable by Player 1 for every e > 0. 

Asymptotic attainability requires that a set is asymptotically reached by the cumulative 
payoff without putting any bound on the time it takes to reach the set. Attainability, on the 
other hand, requires that a set is approximately reached in a bounded time, independent of 
the degree of approximation. Weak asymptotic attainability relaxes both time boundedness 
and the level of the approximation precision. A set Y is weakly asymptotically attainable 
if any neighborhood B(Y, e) around Y can be asymptotically attained, without having a 
universal bound on the time at which this neighborhood is reached. 

Any attainable set is also weakly asymptotically attainable and any asymptotically 
attainable set is weakly asymptotically attainable as well. Analogously to Proposition [TJ 
the set of asymptotically attainable vectors and the set of weakly asymptotically attainable 
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vectors are convex cones. The definition implies that the set of weakly attainable vectors 
is also closed. 

Using Theorem [T] we now show that attainability of a vector does not imply its asymp- 
totic attainability. This implies in particular that these two concepts are not identical. 

Example 2 We provide an example where the vector is attainable but not asymptotically 
attainable. Consider the following game where payoffs are 2-dimensional, each player has 2 
actions, and the payoffs are scalar and given by: 

L R 



1 








-1 



Figure 4: The payoff function in Example [2} 

In this game v\ = for every A £ R. Thus, for every A £ R 2 one has v\ > 0, and 
therefore Theorem [l] implies that the vector is attainable by Player 1 . We argue that is 
not asymptotically attainable by Player 1 . Assume that Player 1 implements a strategy o\ . 
In an initial time interval the strategy o\ plays one of the rows with a positive probability. 
Consider the strategy 02 of Player 2 that plays constantly a column that generates a non- 
zero vector in that initial interval. For instance, if o\ plays the action U with positive 
probability in the initial time interval, then 02 play the action B always. The initial period 
produces a non-zero payoff and this payoff is not diminishing to zero because Player 2 
keeps playing the same column forever. This example shows that is attainable but not 
asymptotically attainable by Player 1. 

We point out that the argument mentioned above shows in fact that is not attainable 
in the corresponding game in discrete time as well. 

The following example shows that a weakly attainable vector need not be attainable. 

Example 3 Consider a two-player game where payoffs are 2-dimensional, Player 1 has 3 
actions, Player 2 has 2 actions, and the payoff function is given by the left-hand side matrix 
in Figure 5. 
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R 
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(1,1) 


(0,1) 


u 


(1- 


6,1-8) 


(-6,1-5) 


M 


(0,0) 


(1,1) 


M 


(" 


-5,-5) 


(1-5,1-5) 


B 


(0,0) 


(0,0) 


B 


(" 


-6, -5) 


(-6,-5) 



The game G The game G — 5(1,1) 

Figure 5: The payoff functions of the games G and G — 5(1, 1) in Example [3j 

The vector (0, 0) is attainable by Player 1, using the strategy that always plays B. The 
vector x := (1,1) is weakly asymptotically attainable according to Definition [3j Indeed, 
given e > consider the strategy erf, with updating times (r-f )keN defined by rf = ke for 
k £ N, that is defined as follows. 

• If the total payoff up to time t\ is not in the set B({1, 1), e), during the time interval 
[t\,t\ +x ) play the mixed action [e(U), (1 - e)(M)\. 

• If the total payoff up to time r-f is in the set B((l, l),s), during the time interval 
[ti,t^ +1 ) play the action B. 

For every t > - one has d( r y t (af , 02), (1, 1)) < e, so that the vector x is indeed weakly 
asymptotically attainable by Player 1. 

The vector x, however, is not attainable by Player 1 (according to Definition [2]) . To 
show that x is not attainable by Player 1 we use Theorem [2] and show that condition B3 
does not hold for x. Indeed, fix 5 > 0, and set q := [(1 - \)(L), f-(-R)]. Let p € A(Ai) 
be arbitrary. If u(p, q) = 5x = (5, 5) for 5 > then necessarily p v = 0. One can verify that 
u(p,q) cannot be equal to 5x for 5 > 5q, and therefore condition B3 does not hold for x. 

Remark 4 The proof of Theorem^shows that every vector x £ M m is attainable by Player 
1 if and only if every vector x £ W 71 is asymptotically attainable by Player 1. Example 
[S] shows that attainability does not imply asymptotic attainability. We are unable to tell 
whether or not asymptotic attainability implies attainability. 

5.3 Alternative strategies in continuous time. 

The strategies we use here are non-anticipating strategies with delay. In these strategies 
the times (r^k^n at which a player observes past play are independent of the play of the 
other player. One could consider a broader class of strategies in which (r^)fc g N are stopping 
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times. In other words, rf +1 is a time that depends on (that is, it is measurable with respect 
to) the information available to player % at time rf, for each k E N. In this type of strategies 
the updating times (r^k^m, are not pre-determined real numbers, as in Definition [l] Our 
results remain valid even if Player 2 is allowed to use a strategy from this broader class of 
strategies. 

5.4 Remaining open problems 

In the applications that we provided, both in control theory and in banking, the target 
sets are often not singletons. All the results above refer to attainability of singletons, and 
did not discuss attainability, asymptotic attainability or weak asymptotic attainability of 
sets. Characterizing when a set of payoffs is attainable (according to these three definitions) 
remains open. We also leave attainability in discrete time and attainability when payoffs 
are discounted for future investigation. 

6 Proofs 

6.1 Proof of Proposition [T] 

The following two lemmas prove the proposition. 

Lemma 1 The set W is a cone. 

Lemma 2 The set W is convex. 

Proof of Lemma [lj Suppose that x E W, and fix /3 > 0. We will show that jjX E W 
as well. 

For every strategy cr, of player i, let erf be the strategy 0{ accelerated by a factor (3. 
That is, (of (a_i))(i) := (<Tj(a_i))(/3t), where a_,(t) = a_,(^i). 

By the definition of attainability, there is T > such that for every e > there is a 
strategy af of Player 1 such that 

ti(z,7*((7i,ff 2 ))<e, Vt>T,Vcr 2 . 

For every strategy a% of Player 2 one has 

7 *(^,(7 2 ) = f u(a s (a^a 2 ))ds 
Jo 

= - / n(a s (a 1 ,^ //3 ))d S (3) 
P Jo 

= (V/OVVi,^). 
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We deduce that 



d{- an T (4,a 2 )) < J, Vt>/3T,V<r 2 , 



and therefore -sx is attainable by Player 1, as desired. ■ 

Proof of Lemma[2j Let x,y G W and let < /3 < 1. We show that /3x+(l—/3)y G VF. 
From Lemma [TJ /3a; and (1 — j3i)y are attainable by Player 1. 

By the definition of attainability there are T x ,T y > such that for every e > there 
are strategies erf and a\ of Player 1 such that 

d( 1 t (af,a 2 ),Px) <e, Vt^T^Va,, (4) 
d( 7 t K,a 2 ),/3y)< e , Vi>T y ,Va 2 . (5) 

Partition the time line [0, oo) into two sets, Ti and T 2 as follows: 

• Ti = U^ [£,£ + P); 

• T 2 = U^ + /3,£+l). 

Thus, each integer interval [£, £ + 1) is partitioned into two sets: one set with Lebesgue 
measure j3 is included in T\ , and the other set with Lebesgue measure 1 — is included in 
T 2 . We construct a strategy cii of Player 1 that uses af on Ti and o\ on T 2 , and show that 
this strategy attains f)x + (l — f3)y up to e. Formally, for each j G {1,2} define (fj(t) to be 
the Lebesgue measure of the set [0, t) DTj. Given a control a_j G C_i define two auxiliary 
controls a ] _ i and a^j by 

ai,.(t) := a-itej 1 ^)) for every i > 0, j G {1,2}. 

The control a^_- consists of that part of a_, that corresponds to times in Tj. The strategy 
o\ is defined by 

' K(a^))(t), iGT l5 
K(a 2 _,))(t), tGT, 



<7i(a_i)(t) := 



Fix a strategy cx 2 of Player 2. 

l T {<y\,a2) = / n(a'(cri,a 2 ))dt 



JO 

= f u(a^ 1 W((j :c ,(j 2i:c ))dt+ / n(a^ 1 W( fT j / ,c7 2i j / ))dt, 

J[0,T)nTi J[0,T)nT 2 

where <7 2j:r and <r 2>2/ are the strategies of Player 2 induced on Ti and T 2 . The right-hand 
side converges to /3x + (1 — /3)y up to e, which completes the proof. ■ 
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6.2 Proof of Theorem Q] 

To see that condition Bl implies condition B2, assume to the contrary that Bl holds but 
condition B2 does not. Then, there is A such that v\ < 0. That is, there is q G A(A2) 
and S > such that (A, u(p, q)) < 5 < for every p G A(Ai). Denote by 02 the stationary 
strategy of Player 2 that plays constantly the mixed action q. This strategy guarantees that 
for every strategy a\ of Player 1 , 

(A, 7 T (a))= / T (A,n,((a*(a)))dt<r5. 
J 

In particular, the distance between Ui{a t {a)')dt and does not go to 0, and therefore the 
vector is not attainable by Player 1, which contradicts condition Bl. 

Suppose now that condition B2 is satisfied. The argument used in the proof of Lemma 
[T] of accelerating the time shows that, given T > 0, if for every e there is a strategy o\ of 
Player 1 such that d(7 (01, 02), 0) < e for every t > T then for every e there is a strategy 
<7i of Player 1 such that d(7*(cri, 02), 0) < e for every i > ^. It follows that to prove that 
condition Bl is satisfied it is sufficient to prove that condition B2 implies that for every 
e > there is T > and a strategy a± of Player 1 such that d(7*(o"i, 02), 0) < e for every 
i > T. 

Consider then the following strategy a\ of Player 1 that depends on a parameter ij > 0. 
The updating times of the strategy a%, (rf )k^, are defined by 

Denote the payoff up to time by Sfc. For < t < r^ +1 we set 01 (t) to be an optimal 
strategy for Player 1 in the game (—Sk, G). That is, u\ is constant in the interval [rf , t^ +1 ); 
in this interval 0"i(t) is equal to a mixed action that guarantees that the payoff and Sk lie on 
different sides of the hyperplane perpendicular to Sk ■ This means that if 02 is the strategy 
played by Player 2, then (Sk,u(a t (a))) < 0, for every t G [rf, Tf +1 ). 
By definition, = 7 T i (o"i,0"2). Thus, 

Sk = S k -i + u(a(t))dt. 
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Consequently, 



\Sk 



-k \ 

|5fc_if + 2( S'fc, / ^(ffi^J.ffs^dt 



15, 



fc-i 



< 



|Sfc- 



< ||5 fc . 




fc-i 
i 



u(ax(t),<T 2 (t))dt 



{S k ,u(a 1 {t),a 2 (t)))dt + 



u(a 1 (t),<T 2 (t))\\dt 



fc-i 
i 



ii(ffi(t),a2(«))dt 



(7) 
(8) 

(9) 
(10) 



Continuing this way, one obtains, ||/Sfc|| 2 < rj 2 Yle=i F ^ ^ r ? 2- Thus, the distance between 
Sk and does not exceed 2r] for any k. 

Note that when k > e^, then vf := £^ =1 | > 7?log(/c) > 1. It means that the cumulative 
payoff at any time T-f that exceeds 1 is within 1r\ from 0. Finally, since the length of the 
time segments [rf , t^ +1 ) is the cumulative payoff up to i > 1 + r/ is within 2ry + -\ from 

0. Since r\ is arbitrary, it follows that the vector is indeed attainable, withj T = 2, by 
Player 1. Condition Bl is therefore established. ■ 



6.3 Proof of Theorem [2] 

Part 1: If the vector x is attainable by Player 1 then is attainable by Player 1. 

Assume to the contrary that the vector x is attainable by Player 1 but is not attainable 
by Player 1. Let To > be arbitrary. Since the vector is not attainable by Player 1, there 
is Eq > such that for every strategy o\ of Player 1 there is a strategy a 2 of Player 2 and 
t > Tq such that d(7 i (cri, a%), 0) > £o- 

Let o\ be a strategy for Player 1 that attains x. In particular, there is T > and a 
strategy a% of Player 1 such that 



d(j t (ai,a2),x) < 



CO 



Va 2 ,Vt > T. 



(11) 



Let a\ be the strategy a\ from time T and on. Formally, let k be the minimal integer such 
that t\ > T. The updating times of a± are (rf)^, and $1(0,2) = crx(a\ o 02) for every 
<i2 G C2 , where i s the play of Player 2 according to 02 up to time T (when Player 1 plays 



4 Recall that in the definition of attainability, see Definition]^] there should be a uniform time T at which 
the cumulative payoff is with e to 0. Here we set T = 2 because it is greater than 1 + 77 and it does depends 
nor on r) either on the strategy employed by Player 1. 
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<7i), and a\ o 02 is the concatenation of a 2 and ai into a log history that starts with a\ and 
continues with 02- 

Let (72 be a strategy for Player 2 and t > To be a positive real such that d(7*(ai, (7*2), 0) > 
£q. Consider now the strategy of Player 2 that plays arbitrarily up to time T, and from 



time T is follows 02. Then d(7*(<Ti, 02), 0) > which contradicts (11). 



Part 2: If the vector x is attainable by Player 1, then condition B3 is satisfied. 

Suppose to the contrary that condition B3 is not satisfied. That is, for every So > 
there is q £ A(A2) such that for every p £ A(Ai) one has u(p, q) 7^ Sx for every S > So- We 
divide the argument into two cases. 

Case A: There is q £ A (^2) such that for every p £ A(Ai), and every 5 > 0, u(p, q) 7^ 5x. 
We show that by playing constantly q (a strategy that we denote by q*) Player 2 can prevent 
Player 2 from attaining x. Let a\ be a strategy for Player 1. Denote by pt the average mixed 
action played by Player 1 up to time t; that is, p l = \ f Q ai(s)ds. We obtain, 7* (01,(7*) = 
tu(pt,q). Thus, 7* (01,0*) is in the cone generated by Ri(q) ■= {u(p,q); p £ A(Ai)}. This 
cone is closed and by assumption, does not contain x. Thus, there is a positive distance 
between x and this cone, implying that 7* (01, q* ) cannot get arbitrarily close to x. This 
contradicts the fact that x is attainable. 

Case B: For every q £ A(^) there is p £ A(A\) such that u(p,q) = 5x, but the S's 
are not bounded away from zero. Thus, for every 5 > 0, there is qs £ A(^) such that 
5 > maxjo 7 ; Bp s.t. u(p,q) = 5'x). We show that for every 5 > 0, if Player 2 plays qs 
all the time (a strategy that we denote by qt), then there is e > such that for every di, 
\h T (o-i,q}) - x\\ < e implies T > 1/(45). 
Fix 5 > 0. Denote 

60 := maxjo'' : 3p s.t. u(p, qs) = 5'x} < 5. 

In particular, 5qx £ R\(qs), and 5'x Ri(q$) for every 5' > So. Let E be the convex hull 
of Ri(qs) and 0. That is, E = conv \ Ri(qg) U {0}^. The set E is convex, compact and it 
does not contain 5'x for every 5' > So. In particular, 25oa; -E 1 . Thus, there is an open 
ball F = B(25ox,rj) which is disjoint of E. By the hyperplane separation theorem there is 
a non-zero vector a £ M m (without loss of generality we may assume that ||a|| = 1) such 
that (e, a) < (/, a) for every e £ E and / £ F. It implies that = (0, a) < (/, a) for every 
f€F. 

We claim that < (x,a). Indeed, if = (x,a), then every / £ F can be expressed 
as / = 25qx + v, where v = v(f) £ B(0,r]). In particular, < (f,a) = (v,a). It follows 
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that (v, a) = for every v G B(Q, r]), which implies that a = 0, contradicting the fact that 
||a|| = 1. 

Suppose that e G Ri(qs) and T • e G B(x,e), with e = (x,a)/2. Then, T • e = cc + z, 
where \\z\\ < e. Thus, (T ■ e, a) = (x + z, a). Since e £ E and 2<5ox G F, 

(e, a) < (2<5ox,a) < (2<5x,a). 

Hence, 

(x + z, a) (g, a) + (z, a) > (x, a) - £ _ 1 
(e,a) ~ 2(&r,a) ~ 2(5x,a) 45' { ' 

Recall that q$ denotes the strategy of Player 2 that plays qs all the time. To show that 

condition Bl holds, that is, that x is not attainable, we need to show that for every T 

there is e > such that for every strategy o\ of Player 1 there is a strategy a% of Player 2 

and t <T satisfying d(-y t (a±,a2),x) > e. Fix a strategy o\ of Player 1, and suppose that 

the cumulative payoff up to T is within e from x. In other words, ||7 T (ci,g|) — x\\ < e. 

Let p T := y Jq ai(s)ds be the average mixed action played by a\ until time T. Thus, 



Tu(pT,qs) = x + z, where ||z|| < e. Letting e = u(pT,qs), we obtain by Eq. (12) that 
T > jj. In words, the time it takes to reach B(x,e) is at least jj. This shows that there 
is no uniform bound on the time at which the total payoff gets close to x. Thus, x is not 
attainable, which contradicts the assumption. 

Part 3: If conditions Bl and B3 hold, then condition B4 holds. 

Let (A, (G — 5x)} be an auxiliary two-player zero-sum one-shot game where the sets of 
actions of the two players are A± and A2, and the payoff function is r(ai, 02) = (A, u(ai,a,2) — 
5x) for every (ai, 02) G A\ x A2 

We prove that there is 5 > such that for every vector A G M. m one has val((A, (G — 
fix)}) > 0. Theorem [l] would then imply that the vector is attainable in the game 
(A, (G — 5x)), so that condition B4 would hold. 

To this end we show that there is 5 > such that for every A G M m there is p G A(^4i) 
such that for every q G A(A2), (A, u(p, q) — 5x) > 0, or equivalently, (A, u(p, q)) > (A, 8x). 

Fix A G R m . Assume first that (A, 2) < 0. By condition Bl we have that v\ > 0, 
and therefore there is p G A(Ai) such that for every q G A(A2), (X,u(p,q)) > 0. Hence, 
(A, u(p, q)} > 5{\, x) = (A, 5x) for every 5 > 0, as desired. We can therefore assume that 
(X,x) > 0. From condition B3 we know that for every q G A(A2) there is p G A(A±) 
such that u(p, q) = 5x with 5 bounded away form 0. Therefore, there is 5q such that for 
every q G A(^2) there is p G A(Ai), such that (A, u(p, q)) = (X,5 q x) > (X,5qx). By the 
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minmax theorem, there is p G A(^4i) such that for every q £ A(A2), (X,u(p,q)) > (A, Sox), 
as desired. 

Part 4: If conditions Bl and B4 hold then the vector x is attainable by Player 1. 

Let 77 > and let e > satisfy < r/. By condition B4 the vector is attainable by 
Player 1 in the game (G — Sx), so that there is T > such that for every e > there is a 
strategy o\ of Player 1 satisfying 

d(7(G-fa)K ff 2),0)<e, V<j 2 ,Vi>T. (13) 

By Eq. ([3]), for every /3 > one has 

d (^7fG_ fa) K 1//3 ^2),0)<e, Va 2 ,Vt>T. (14) 



It follows that 
so that 



d(^ 5x) (vl /(3 ,°2),d)<pe, Va 2 ,Vi>T, (15) 

d(ia-SxM /P ^2),0) </3e, Va 2 ,Vi>/3T, (16) 
Setting p = jFp we deduce that 

d(7(G-fa)(^r^2),6) < ^, Va 2 ,Vt > i (17) 

Setting t = t in (17) it follows in particular that 

rfCTG^f,^)^) < ^ < 77, Va 2 . (18) 

The strategy af T ensures that the payoff at time 4 is close to x. It follows that the strategy 
of Player 1 that follows a{ T up to time \ , and follows a strategy that attains in the game 
G thereafter, is a strategy that attains x in G. ■ 

6.4 Proof of Theorem |3] 

We first prove that condition C2 implies condition CI. If condition C2 holds then condition 
B2 holds as well. By Theorem [l] the vector is attainable by Player 1. By Theorem [ij to 
prove that condition CI holds, it is sufficient to prove that condition B4 holds for every 
x £ R m . 

Assume then to the contrary that condition B4 does not hold for some x £ M m . Then, 
for every 5 > the vector is not attainable by Player 1 in the game (G — Sx). By Theorem 
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[T] this implies that there is Xs € M m such that v\ s (G — 5x) < 0. Since this inequality does 
not hold for v*(G — 5x) = for every 5 > 0, we can assume w.l.o.g. that IjA^H = 1 for every 
5 > 0. Since v\ s (G — 5x) < 0, there exists qs € A(^2) such that 

(u(p, Qs) - Sx, X s ) < 0, VpeA(Ai). 

Take a subsequence (Sk)keN that converges to such that the sequence (qs k )keN converges 
to a limit q* e A(^) and the sequence (Xs k )km converges to a limit A* E M m . Note that 
|| A* || = 1, so that A* j£ 0. Then 

(u(p,q*),X*) < 0, VpeA(^i). 

Ii implies that v\ t < 0, contradicting C2. 

It remains to show that condition CI implies C2. Assume that the payoff function u 
does not satisfy condition C2. Then, there is A S M m such that v \ < 0. It follows that there 
is q £ A(A 2 ) such that D 2 (q) := p G A(^4i)} C i7 := {x: (A, x) < 0}. Therefore, 

if Player 2 employs the strategy that always plays q, the total payoff is in H. Thus, every 
payoff vector in the complement of H is not attainable by Player 1. ■ 
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